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IS.  ABSTRACT 

This  paper  reports  on  the  design  of  a  program  visualization  (PV) 
environment,  intended  to  provide  lifecycle  support  for  software  developme 
flie  PV  environment  will  capitalize  on  recent  progress  in  the  graphical 
representation  of  information  and  low*cost  color  graphics,  to  provide 
designers  and  programmers  with  both  static  and  dynamic  (animated) views 
of  systems.  The  aim  is  to  support  maintainers  of  large  (10**6  lines  of 
code),  complex  software  systems.  The  PV  system  will,  in  effect,  ‘open 
the  side  of  the  machine*  to  permit  users  to  look  inside  and  watch  their 
programs  run.  In  this  paper,  we  survey  categories  of  program  illustratio: 
and  then  present  and  motivate  the  design  philosophy  that  we  are  pursuina 
for  the  PV  environment.  * 
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1.  MAJOR  TECHNICAL  RESULTS 


In  late  spring  of  1981  we  wrote  a  paper  outlining  our 
goals  for  the  Program  Visualization  (PV)  System  and  the 
approach  we  are  taking  to  meet  these  goals.  This  paper, 
"An  Integrated  Environment  for  Program  Visualization",  was 
delivered  at  the  the  IFIP  VG  8.1  Working  Conference  on 
Automated  Tools  for  Information  Systems  Design  and 
Development,  New  Orleans,  26-28  January,  1982.  The  paper, 
a  copy  of  which  is  enclosed  with  this  report,  was  pub¬ 
lished  in  Schneider  and  Wasserman  (eds.),  Automated  Tools 
for  Information  Systems  Design,  North-Holland  Publishing 
Co.,  1982. 

The  bulk  of  198I  was  spent  designing  the  Program 
Visualization  (PV)  System  and  identifying  certain  key 
classes  of  images  that  the  system  should  support.  Partic¬ 
ular  attention  was  paid  to  the  design  of  dynamic  Images  to 
illustrate  programs  as  they  run. 

This  phase  of  the  project  culminated  in  a  videotape 
produced  in  December  1981.  The  tape  was  edited  into  a 
more  concise  version  in  January  1982,  and  narration  was 
added.  Graphics  for  the  tape  were  developed  using  the 
Paint  and  Animation  subsystems  of  the  Spatial  Data  Manage¬ 
ment  System.  The  animated  images  on  the  tape  were  mock- 
ups  in  the  sense  that  they  were  not  yet  driven  by  underly¬ 
ing  software.  The  tape  shows  animated  graphic  depictions 
of  control  flow  and  data  updates.  An  important  feature  of 
the  Images  that  were  designed  for  PV  is  that  they  present 
a  selection  of  levels  of  detail,  giving  the  programmer  a 
choice  of  either  an  overview  or  more  detailed  views  of  the 
system  he  or  she  is  building. 

A  copy  of  the  1982  Program  Visualization  videotape  is 
enclosed  with  this  report. 

Spring  1982,  the  final  portion  of  the  time  period 
covered  by  this  report,  was  spent  doing  system  design  and 
planning  for  the  implementation  effort  that  began  in  May 
of  1982. 
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2.  TECHNOLOGICAL  SIGNIFICANCE 


He  expect  on-line,  Interactive  graphics  to  have  a 
profound  inpact  on  software  development,  analogous  to  the 
way  that  word  processors  have  affected  the  production  of 
text.  With  respect  to  static  graphics,  the  multi¬ 
dimensional  graphic  information  structure  that  we  are 
developing  for  PV  will  allow  the  programmer  to  move  easily 
between  pieces  of  related  Information  (e.g.  requirements 
and  system  structure).  With  respect  to  dynamic  graphics, 
PV's  animated  views  of  program  execution  can  help  program¬ 
mers  achieve  a  deeper  and  more  accurate  understanding  of 
the  behavior  of  their  programs. 
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3.  PRESENTATIONS  AND  PUBLICATIONS 


December  1981 

Chriatopher  Herot,  Mark  Friedell,  and  Diane 
Smith  took  part  in  a  DARPA  conference  organized  by 
Craig  Fields  of  the  System  Sciences  Division.  Copies 
of  the  presentations  were  compiled  in  "DARPA  Confer¬ 
ence  on  Computer  Software  Graphics.  Key  West  Florida. 
Dec.  13-15  1981." 


January  1982 

Christopher  Herot  and  Gretchen  Brown  took  part 
in  the  IFIP  VG  8.1  Working  Conference  on  Automated 
Tools  for  Information  Systems  Design  and  Development, 
New  Orleans,  26-28  January,  1982.  The  paper 
presented  at  this  conference,  "An  Integrated  Environ¬ 
ment  for  Program  Visualization",  appeared  in 
Schneider  and  Wasserman  (eds.),  Automated  Tools  for 
Information  Systems  Design,  North-Holland  Publishing 
Co.,  1982. 


February  1982 

Christopher  Herot  gave  a  presentation  on  PV  at  a 
meeting  of  the  Northeastern  ACM  Chapter  on  February 
18. 
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M  IITlGMTtO  ENVIRONMENT  TOR  PROGRAM  VISOALUATION* 


Christopher  r.  Barot,  Gratcban  p.  Brown 
Richard  T.  Carling,  Mark  rriadall 
David  Kraalich,  Ronald  N.  Baecker** 

Computer  Corporation  of  America 
575  Technology  Square 
Cambridge,  Raaaacbuaetta,  DBA 


This  paper  reporta  on  tbe  deaign  of  a  program  viaualiaation 
(IV)  an vi ronnant,  intended  to  provide  lifecycle  aupport  for 
aoftvare  development.  Tbe  W  environment  will  capital iia  on 
recent  progreaa  in  tbe  graphical  repreaentation  of  intonation 
and  low-coat  color  grapblca,  to  provide  designers  and  program- 
aera  witb  both  static  and  dynaaic  (animated)  views  of  systems. 
Tbe  aim  is  to  aupport  maintainors  of  large  (10**S  lines  of 
code)*  complex  software  systems.  Tbe  FV  system  will,  in 
effect,  *open  tbe  side  of  tbe  machine*  to  penlt  users  to  look 
inside  and  watch  their  programs  cun.  In  this  paper,  we  survey 
categories  of  program  illustrationa  and  then  present  and 
motivate  tbe  design  philosophy  that  we  are  pursuing  for  tbe  FV 
environment. 


1.  INTRODUCTION 

Graphical  representations  have  demonstrated  their  utility  in  a  wide 
variety  of  design  and  implementation  activities  as  a  means  of  illus¬ 
trating  complex  relationships  among  components  of  systems.  It  would 
be  Inconceivable  to  build  a  ship,  airplane,  factory,  or  piece  of 
electronic  equipment  without  the  use  of  diagrams.  These  illustra¬ 
tions  can  capture  essential  features  while  suppressing  extraneous 
detail,  and  they  can  often  be  understood  more  readily  than  ordinary 
text.  While  such  illustrations  find  widespread  use  in  computer  pro¬ 
gramming,  they  are  almost  always  msnually  generated,  making  produc¬ 
tion  and  revision  laborious. 


•  This  research  was  supported  by  the  Defense  Advanced  Research  Fro- 
Jeet  Agency  of  the  Department  of  Defense  and  was  monitored  by  tbe 
Office  of  Naval  Research  under  Contract  No.  M0O14-SO-C-MS3.  The 
views  and  conclusions  contained  in  this  document  are  those  of  tbe 
authors  and  should  not  be  Interpreted  as  necessarily  representing 
the  official  pelicles,  either  expressed  or  implied,  of  tbe  Defense 
Department,  the  Office  of  Naval  Research,  or  the  D.S.  Government. 
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Om  result  of  manual  graphics  gsnsration  is  that  diagrams  hast  a 
tendency  to  become  obsolete  as  the  software  they  describe  is  imple- 
aented  and  changed.  A  second  result  is  that  the  full  variety  and 
utility  of  graphical  lmagea  raaains  unesploited.  Third,  the  lack  of 
tools  for  creating  animated  images  restricts  the  ability  to  illus¬ 
trate  an  essentially  dynamic  process  such  as  a  computer  program. 
This  is  not  to  say  that  automated  graphical  representations  have 
been  totally  neglacted.  There  has  boon  some  success  in  automating 
production  of  static  representations  of  programs  (e.g.,  (>,14,141). 
The  idea  of  using  computer-generated  images  to  visualise  the 
dynamic  behavior  of  programs  was  developed  in  the  earliest  days  of 
computer  graphics  (110,20,341),  and  dynamic  visualisations  have 
received  continued  attention  (e.g.,  13, 4,0, 0,15,21, 22, 401).  Only 
recently,  however,  have  hardware  and  software  advances  been  made 
which  would  allow  automated  production  of  both  static  and  dynamic 
illustrations  to  become  cost-effective  for  a  broad  range  of  applica¬ 
tions.  In  addition,  significant  research  ramalna  to  be  done  before 
diverse  graphical  representations  can  be  successfully  integrated 
within  a  single  environment. 

The  Computer  Corporation  of  America  is  in  the  first  year  of  a  three 
year  effort  to  design  and  implement  a  program  visualisation  (PV) 
system.  The  verb  viauai means  *to  see  or  form  a  mental  image 
of”.  NO  want  to  aid  programmers  in  the  formation  of  clear  and 
correct  mental  images  of  the  Structure  and  function  of  programs. 
Program  visualisation  has  groat  premise  for  all  stages  of  the 
software  lifecycle.  Our  research  will  focus  on  illustrating  the 
dynamic  behavior  of  programs,  which  we  aspect  to  be  of  most  use  in 
testing,  debugging,  maintenance,  and  training,  we  want  the  PV  sys¬ 
tem  to,  in  effect,  *open  the  side  of  the  machine”  ao  that  the  user 
can  form  an  accurate  model  of  the  program. 

Tbe  PV  system  will  provide  an  integrated  graphics  environment,  capi¬ 
talising  on  recent  progress  in  the  graphical  representation  of 
information  and  lew-cost  color  graphics.  The  aim  is  to  support 
builders  and  maintainers  of  large  (10**<  lines  of  code),  eonples 
software  systems.  (We  are  escluding,  however,  real  time  systems.) 
This  tool  is  targeted  primarily  for  use  with  programs  written  in  Ada 
(1),  the  proposed  standard  DoD  language. 

The  system  which  is  envisioned  will  provide  individuals  who  must 
build  and  maintain  a  camples  software  system  with  access  to  a 
variety  of  graphical  representations.  These  will  include  static 
descriptions,  such  as  module  hierarchies  and  requirements  specifica¬ 
tions,  and  dynamic  illustrations,  such  as  procedure  activations  and 
storage  allocations.  It  will  be  possible  to  display  several  dif¬ 
ferent  representations  of  the  sane  portion  of  a  system  (or  the  same 
representation  of  several  different  portions)  simultaneously  through 
the  use  of  multiple  screens  or  multiple  viewports  am  erne  screen. 
Since  we  are  focusing  on  the  dynamic  behavior  of  programs,  we  will 
make  ostenslve  use  of  computer  animation. 
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Tbit  paper  ia  intended  to  initiate  a  debate  concerning  the  charac¬ 
teristics  of  a  good  graphics  environment  for  software  production. 
Section  2  surveys  s  nuaber  of  cstegories  of  information  that  merit 
graphical  presentation.  The  nuaber  and  variety  of  these  categories 
suggests  that  fuller  esploitation  of  graphics  can  profoundly  influ¬ 
ence  software  production,  lust  as  test  editing  facilities  have 
changed  the  way  that  papers  are  written.  The  challenge  that  we  see 
is  to  encompass  the  voluaw  and  diversity  of  this  information  within 
a  coherent  conceptual  framework.  Section  3  espounds  a  design  philo¬ 
sophy  for  PV.  Ne  address  there  three  significant  problems: 
Integrating  the  categories  of  illustrations  identified  in  Section  2, 
enhancing  graphical  facilities,  and  instrumenting  the  program. 
Implementation  plans  for  the  PV  system  are  discussed  in  Section  4, 
with  a  summary  in  Section  5. 


2.  CATEGORIES  OP  VISOJU.IIATIORS 

It  is  our  claim  that  although  graphics  has  long  been  a  tool  in  pro¬ 
gram  development  and  documentation,  the  full  power  of  graphics  has 
yet  to  be  acknowledged  or  esplolted.  This  section  lists  ten 
categories  of  program  illustrations  that,  together,  can  be  of  use 
throughout  the  software  lifecycle.  These  categories  were  one  result 
of  a  sis  month  study  conducted  to  provide  a  conceptual  framework  for 
program  visualisation.  Some  categoriea  of  illustrations  have 
already  been  well  esplored  with  reapect  to  programs,  and  the  PV 
environment  will  draw  on  this  work  directly.  Other  categories  have 
been  less  thoroughly  asplored,  and  suggestions  for  new  directions 
are  included  here.  The  ten  categories  are: 

1.  System  requirements  diagrams 

2.  Program  function  diagrams 

3.  Program  atructure  diagrams 

4.  Communication  protocol  diagrams 

5.  Composed  and  typeset  program  test 

C.  Program  comments  and  commentaries 

7.  Diagrams  of  flow  of  control 

t.  Diagrams  of  structured  data 

P.  Diagrams  of  persistent  dsts 
10.  Diagrams  of  the  program  in  the  host  environment 

Many  of  these  categories  can  apply  to  either  JJhs  program  or  its 
specific  mHmhwi.  Moreover,  illustrations  can  be  either  static 
or  dynamic.  static  illustrations  portray  the  program  at  some 
instant  of  osecutlon  time,  or  they  portray  those  aspects  of  a  pro¬ 
gram  which  are  invariant  over  a one  interval.  Dynamic  illustrations 
portray  the  progress  of  an  assenting  program. 

We  shall  now  look  at  each  of  the  tea  categories  of  illustration  in 
more  detail. 
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2.1  Systea  Raqulreaents  Diagraas 

A  computer  prograa  always  aaiata  aa  pact  of  soae  lacgat  systea  (not 
nscaaaarily  a  fully  autoaated  ona) .  Therefore,  PV  tools  oust  assist 
In  tbe  portrayal  of  the  function  and  atructuca  of  that  systsn.  The 
tools  should  also  aid  in  tha  spacification  of  tha  constraints 
iaposed  by  tha  systra  on  tha  progtan. 

One  vary  powerful  nathod  of  describing  syataa  structure  is  tba  IDEF 
or  SADT  technique,  developed  by  AofTech  1221 .  An  IDEF  nodal  is  a 
graphical  representation  of  a  systea  in  tarns  of  its  subsysteas  and 
in  tarns  of  tha  data  and  control  flow  that  links  than  together.  The 
nathod  deals  with  tha  hierarchic  nature  of  seat  ayataas  quite  natur¬ 
ally,  and  it  provides  a  aethodology  for  organising  the  bookkeeping 
associated  with  large  eoaples  systea  descriptions. 

A  coaplete  raquiraaents  specification  also  contains  constraints  on 
the  prograa's  design,  constraints  such  as  eaecutlon  speed,  prograa 
else,  user  interface  style,  iapleaentatlon  vehicle,  iapleaentation 
cost,  and  the  like.  We  are  investigating  whether  there  is  any  sig¬ 
nificant  role  for  graphics  in  describing  these  latter  specifica¬ 
tions. 

2.2  Prograa  Function  Diagraas 

■What  does  the  prograa  do7a  is  usually  the  first  question  that  one 
asks  about  a  prograa.  Pot  aaay  types  of  prograas,  prograa  function 
can  be  viewed  as  a  napping  froa  prograa  input  to  prograa  output. 

we  can  talk  about  the  relationship  of  prograa  input  to  output  in  two 
vary  different  waysi  a  stataaant  of  the  prograa's  function  in  gen¬ 
eral  teras  or  an  enuae ration  of  a  nuaber  of  input-output  pairs.  Tbe 
former  is  a  wore  powerful  and  useful  description,  but,  because  it  is 
an  abstraction,  it  poses  difficult  problems  for  graphical  represen¬ 
tation. 

Graphical  portrayal  of  aaaple  behaviors  is  s  aoch  more  straightfor¬ 
ward  proposition.  Thus  one  approach  to  tbe  visualisation  of  prograa 
function  is  to  provide  a  'casebook*  through  which  the  user  can 
browse,  inducing  a  nodal  of  what  the  prograa  is  supposed  to  do  by 
seeing  what  it  actually  does  on  a  carefully  selected  set  of  aaaple 
inputs.  the  choice  of  these  saaple  inputs  can  have  a  significant 
affect  on  the  utility  of  this  technique.  Por  esaaple,  in  under¬ 
standing  a  factorial  function,  laportant  values  and  classes  are  0, 
1,  positive  integers,  negative  integers,  resls,  and  aon-nuaerics. 

2.2  Prograa  Itrectare  Diagraas 

**ow  is  tha  prograa  organised?*  is  often  the  second  question  that 
ana  asks  about  a  prograa.  Prograa  structure  has  a  wall-developed 
history  of  graphical  notation.  Relatively  recant  asaaples  are  tha 
■IPO  (lierarchy  plus  Input-Procass-Output)  technique  (14,221,  which 
integrates  structure  diagraas  and  function  diagraas,  and  the  ceapo 
site  design  structural  notation  (231. 
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2.4  Co— unlcatlon  protocol  Diagrams 

One*  it  1*  known  bow  a  prograa  ia  divided  into  lta  component  parts, 
it  ia  useful  to  know  bow  tboaa  parts  co— unieato.  This  ia  espe¬ 
cially  inportent  whan  the  prograa  consists  of  aany  processes  running 
on  one  or  note  processors,  bn  illustration  of  tbe  potential  paths 
for  data  flow  between  nodules  can  be  displayed  as  part  of  another 
diagraa.  For  instance,  tbe  prograa  structure  diagraa  can  be  over¬ 
laid  with  lines  showing  tbe  data  paths  between  aodules.  By  using 
this  technique  dynamically,  the  actual  flow  of  data  can  be  aonitored 
during  esecution.  for  esaaple,  the  8DD-1  distributed  data  base  sys¬ 
tem  (301  at  CCA  employs  a  color  graphics  terminal  to  show  data 
transfers  between  sites  on  the  Arpanet  as  they  occur.  This  monitor¬ 
ing  technique  has  proven  useful  both  as  a  deaonstration  and  a  debug¬ 
ging  aid.  Figure  1  is  a  reproduction  of  a  typical  800-1  illustra¬ 
tion,  streaalined  for  the  purposes  of  this  paper,  fxcept  where  oth¬ 
erwise  noted,  the  figures  in  this  paper  are  all  banddrawn  aockups. 

2.5  Composed  and  Typeset  Prograa  Test 

The  central  activity  in  the  visualisation  of  programs  has  always 
been  the  reading  of  prograa  code.  While  alternative  graphical  tech¬ 
niques  are  proposed  here,  there  will  still  be  cases  where  code  must 
be  esaained.  This  task  can  be  aade  significantly  easier  than  it  is 
at  present.  Some  relevant  typographic  tools  that  can  be  applied  to 
tbe  display  of  programs  arei 

1.  The  use  of  tyJBflfl MBfaiC  faitraiChlkS  for  distinguishing  a 
prograa* s  constituent  elements  that  belong  to  various  syntactic 
or  semantic  categories.  Typographic  hierarchies  are  inpleaented 
by  the  consistent  and  controlled  use  of  a  variety  of  type  fonts, 
type  styles  within  a  font  (bold,  condensed,  italic,  etc.),  and 
point  sises. 

2.  The  use  of  a  rich  eynboi  repertoire  employing  a  wider  range  of 
symbols  and  colors  than  are  currently  uaed.  (Gutenberg  had  more 
than  300  symbols  in  his  type  case.) 

3.  The  use  of  cmpaeiMm  and  iAyBUt  to  facilitate  the  structured 
perusal  of  a  program's  constituent  substructures.  Layout  con¬ 
ventions  include  the  use  of  indentation,  borlsontal  paragraph¬ 
ing,  vertical  paragraphing,  pagination,  footnotes,  marginal 
notes,  and  page  beadera.  The  use  of  coaputer  graphics  also  per- 
alts  dynamic  techniques  such  as  colored  highlighting  over 
selected  or  active  portions  of  prograa  test. 


Theae  technique a  can  be  applied  both  to  produce  displays  on  high 
resolution  terminals  and  to  produce  bard  copy  on  a  demand  basis. 
Figure  2  shows  the  application  of  some  of  these  techniques  to  a  sub¬ 
routine  written  ia  C. 
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Figure  1.  A  snapshot  of  a  dynamic  Illustration  of  cowwuni  cation 
in  8DD-1(  a  distributed  data  base  systea.  Four  sites  are  shown. 
Transaction  nodules  (TMs)  send  transactions  (arrows)  to  data 
nodules  (DNs) . 


Figure  2.  Oee  of  layout  and  typography  for  code  and  coaaents. 
The  aubroutine  adas_copy  ia  written  in  C.  Typography  ia  uaed  to 
eaphasite  important  lines  of  the  subroutine,  and  coaaenta  appear 
on  the  right  in  blocka  to  indicate  their  acope. 
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2.C  frofCM  Comments  and  Commentaries 

Program  comments,  often  known  no  Internal  documentation,  ore  analo¬ 
gous  to  the  footnotes  and  marginal  notes  of  conventional  literacy 
expression.  Program  commentaries,  often  known  as  esternal  documen¬ 
tation,  are  analogoua  to  prefaces,  introductions,  postscripts,  and 
critical  expository  analyses.  Both  comments  and  commentaries  are  an 
important  part  of  conventional  programming  discipline,  yet  they  fall 
far  short  of  attaining  their  ultimate  potential.  Bow  can  they  be 
improved? 

The  greatest  potential  for  Improvement  comas  from  an  area  for  which 
there  is  no  technological  fix,  that  is,  the  ability  of  programmers 
and  documentation  specialists  to  write  in  English  with  clarity  and 
Consistency.  An  integrated  graphics  system  can,  however,  provide 
significant  aid  to  the  programmer  in  other  respects.  If  a  PV  system 
supplies  a  variety  of  graphical  cepreaentations  to  illustrate  dif¬ 
ferent  facets  of  a  program,  then  there  will  be  much  leas  need  for 
pure  text  comments  and  commentary.  Comments  end  commentary  will 
tend  to  be  limited  to  very  general  remarks  on  the  one  hand,  and  to 
very  particular  observations  (e.g.,  noting  exceptions  and  special 
cases),  on  the  other.  Comments  and  commentary,  in  their  reduced 
role,  can  also  benefit  from  the  typesetting  and  composition  tech¬ 
niques  discussed  above  for  program  text.  (See  Figure  2,  for  the  use 
of  lines  end  spacing  to  delineate  the  scope  of  comments.)  finally, 
completeness  and  consistency  of  comments  and  commentary  can  be  sup¬ 
ported  (although  never,  of  course,  assured)  by  the  programming 
environment. 

2.7  Dlagrame  of  Plow  of  Control 

"What  happens  when  the  program  executee?"  is  another  important  ques¬ 
tion  about  a  program.  "In  what  order  do  things  happen?"  is  one 
eubquestion,  with  diagreme  of  flow  of  control  as  a  relatively  fami¬ 
liar  means  of  conveying  the  answer.  (Another  eubquestion,  apropos 
of  the  effect  of  program  execution  on  underlying  data,  is  discussed 
in  Section  2.1.) 

Plow  charts,  Massl-fhneldernan  Diagrams  (g,24).  Software  Diagram 
Descriptions  (IS),  CJUEUPMWTS  IS),  and  others  are  a  beginning,  but 
they  portray  only  the  static  structure,  not  the  actual  flew  of  con¬ 
trol  during  program  execution.  Some  formalisms  already  provide  a 
means  to  indicate  flow  of  control  (e.g.,  the  tokens  in  Petri  nets 
120).  It  is  only  e  short  distance  to  computer  animations  of  flew 
of  control,  with  tokens,  highlighting,  and/or  color  changes  used  to 
denote  the  current  state  of  the  execution. 

2.S  Diagreme  of  Structured  Data 

"What  happens  when  the  program  executee?"  can  also  be  answered  in 
terms  of  the  data  base  upon  which  the  program  is  operating.  This 
data  base  includes  the  program  input  at  the  initiation  of  execution 
and  the  program  output  at  the  termination  of  execution.  The  data 
base  also  Includes  the  variables  that  ere  the  raison  d'etre  of  a 
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pc 04 cm,  such  as  the  date  being  sorted  by  a  quicksort,  and  the  vari¬ 
ables  that  ara  Incidental  to  the  program's  function,  that  is,  the 
artifacts  of  a  particular  piece  of  code  or  programming  technique. 
The  major  difficulty  in  representing  this  information  results  fro* 
the  sise  and  complexity  of  the  data  bases  of  most  interesting  pro¬ 
grams.  It  is  for  this  reason  that  we  have  spoken  of  ‘diagrams  of 
structured  data.*  It  is  only  through  structuring  the  complexity 
(either  at  the  design  stage  or  under  interactive  programmer  control) 
that  we  are  able  to  comprehend  it  and  master  it. 

An  appropriate  illustration  of  a  program's  data,  updated  dynaaiically 
during  program  execution,  gives  us  the  feeling  of  looking  into  the 
machine  and  seeing  the  program  running.  Baecker'a  pilot  film  on 
sorting  algorithms  (3),  and  the  work  of  Knowlton  (201,  Hopgood  US), 
Myers  122),  and  others,  have  vividly  demonstrated  the  power  of  this 
technique.  Figure  3  shows  one  possible  layout  for  displaying  a  two 
dimensional  array.  The  visualisation  of  structurad  data  appears  to 
be  one  of  the  most  tractable  and  powerful  of  the  approaches  we  have 
presented. 

2.1  Diagrams  of  Persistent  Data 

An  important  category  of  structured  data  is  that  which  remains  in 
the  computer  system  after  the  program  has  ceased  execution,  aa 
occurs  in  a  data  baae  management  system.  Since  this  data  is  often 
several  orders  of  magnitude  larger  than  the  memory  capacity  of  the 
computer,  different  techniques  are  required  to  visualise  it.  For¬ 
tunately,  the  data  baae  community  baa  developed  a  rich  set  of  sym¬ 
bols  which  can  serve  aa  a  starting  point  in  visualising  persistent 
data  («.«.,  12,32)).  In  addition,  the  Spatial  Data  Nanagament  Sys¬ 
tem  (SDK SI  111)  developed  at  CCA  has  demonstrated  the  feasibility  of 
using  graphics  to  access  persistent  data.  Figure  4  shows  layouts 
takan  from  an  actual  SDNS  interface  to  a  data  baae  of  information 
about  ships. 

2.10  Diagrams  of  the  Program  in  the  Boat  environment 

There  is  a  collection  of  information  about  programs  that  is  typi¬ 
cally  available  in  varioua  forma  from  operating  systems,  but,  just 
as  typically,  the  information  is  presented  in  a  rather  dense  and 
detailed  tabular  fora.  This  information  includes  the  files  in  which 
program  parts  are  stored,  their  site,  age,  and  ownership.  For  a  pro¬ 
gram  activation,  performance  and  timing  information  is  important. 
The  type  and  percentage  of  resources  currently  in  use,  priority 
under  which  the  program  is  running,  and  the  average  response  time 
for  interactions  are  all  commonly  of  interest. 

Much  can  be  done  to  enhance  the  presentation  of  this  information. 
Percentages  csa  bo  displayed  aa  plea,  histograms,  etc.  Color  coding 
can  be  used  to  point  up  similarities  among  pieces  of  information  or 
to  highlight  particular  properties  of  interest.  Finally,  the  typo¬ 
graphic  hierarchies  discussed  for  program  text,  comments,  and  com¬ 
mentaries  can  play  a  role  here  as  well. 
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Figure  3.  Duplay  of  the  contents  of  an  array  called  AVALS. 
Array  is  set  in  s  window  and  values  can  be  exaained  by  pressing 
arrows  to  scroll  up  or  down. 
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Figure  4.  Photos  froa  SDKS  showing  categorisation  of  ship 
inforaatlon  by  country  (borisontal)  and  by  ship  type  (vertical). 
Detail  photo  corresponds  to  highlighted  portion  of  photo  at  top. 
Shape  of  the  ship  indicates  ship  type,  and  text  ia  identifica¬ 
tion  and  conaand  inforaatlon  taken  froa  the  data  baae.  Addi¬ 
tional  inforaatlon  conveyed  by  color  coding  is  visible  as  grey 
scale  in  these  pictures. 
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3.  A  PV  DESIGN  PHILOSOPHY 

The  previous  section  enumerated  a  "shopping  list"  of  ten  categories 
of  graphical,  and  often  dynamic,  information  about  a  program.  It 
should  be  clear  from  this  list  that  graphics  already  plays  a  signi- 
ficant  role  in  the  software  lifecycle;  it  should  also  be  clear  that 
the  possibilities  of  graphics  have  only  begun  to  be  exploited.  Much 

work  has  been  done  in  developing  techniques  : U “'^ir^ne^to^u^ 
at  various  levels,  but  comparatively  little  has  been  done  to  auto¬ 
mate  these  techniques  to  give  programmers  a  clear  concept  of  the 
dynamics  of  the  program.  This  latter  goal  is  the  focus  of  our  work. 
The  PV  system  will  "open  the  side  of  the  machine"  and  give  the  user 
the  feeling  of  looking  in  and  seeing  the  program  run.  The  pnma  y 
impact  of  such  a  focus  is  on  the  testing,  debugging,  maintenance, 
and  training  phases  of  a  program's  lifecycle.  We  expect  good 
dynamic  program  illustrations  to  greatly  enhance  the  programmer  s 
effectiveness  in  each  of  these  areas. 

Before  we  can  "open  the  side  of  the  machine",  there  are  a  number  ot 
problems  that  must  be  solved.  W.  have  chosen  three  of  them  to 
explore  in  depth.  These  are: 

—  Integrating  information: 

Bow  can  the  ten  categories  of  illustrations  and  text  be  com¬ 
bined  meaningfully? 

--  Enhancing  graphics  facilities: 

Bow  can  we  provide  the  PV  user  with  very  high  level  graphics 

facilities? 

—  Instrumenting  programs: 

Bow  can  programs  be  instrumented  to  permit  access  to  the 
relevant  dynamic  infraction  without  requiring  extensive 
programmer  intervention? 

Other  research  problems  exist,  but  we  feel  ‘“at,  qlv.n  the  current 
state  of  knowledge,  these  three  are  the  most  immediate.  In  partic 
lar,  a  whole  series  of  research  problems  can  be 

heading  of  program  soundness:  how  can  the  correctness  of  the  design 
and  the  code  be  guaranteed?  While  this  is  an  important 
area,  it  is  one  that  is  already  receiving  a  significant  amount  of 
attention  in  a  variety  of  contexts.  Program  **  4 

relatively  well  developed  area,  and  promising  work  is  also  being 
don.  in  program  understanding  <..,..117,1.1)  --  Tt 

gram  templates  to  avoid  errors  (e.g., 

information  about  a  program  as  a  oata  base  has  been  coupled  with 
enforcement  of  standard.  (e.g.,  in  CADES  1251)  and 

a  In  the  DREAM  System  1281).  This  treatment  also  opens  the  way 
f Of9 'application  of  -ore  general  work  on  the  inintle  iHturlty  of 
data  bases.  While  the  PV  system  will  Include  some  checking,  cross 
referencing,  and  decision  support,  insuring  the  soundness  of  the 
information  is  not  a  current  research  focus. 
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In  this  section,  we  present  and  aotivate  a  PV  design  approach  that 
addresses  the  three  problems  of  integration  of  information,  enhance¬ 
ment  of  graphics  facilities,  and  program  instrumentation. 

3.1  Integrating  Information 

We  expect  a  large  volume  of  diverse  information  to  be  involved  in 
the  visualization  of  a  program.  This  is  true  first  because  the  PV 
environment  will  support  the  production  of  large  programs  and, 
second,  because  it  will  provide  a  variety  of  graphical  representa¬ 
tions.  The  challenge  is  to  integrate  the  various  techniques  identi¬ 
fied  in  Section  2  into  a  coherent  whole. 

The  organization  of  program  information  (‘program  space*)  that  we 
are  currently  exploring  is  the  placement  of  visualizations  in  a 
hierarchy  of  two-dimensional  surfaces.  The  levels  in  this  hierarchy 
would  match  the  structure  of  the  program  under  investigation.  As 
one  descended  in  the  spatial  hierarchy,  one  would  view  successively 
more  detailed  representations,  for  instance  viewing  increasingly 
lower-level  modules.  For  each  node  in  the  hierarchy,  the  surface 
would  contain  examples  of  some  or  all  of  the  illustration  categories 
described  in  Section  2.  (This  includes  dynamic  illustrations,  which 
can  be  thought  of  as  movies  embedded  in  surfaces,  runable  under  user 
control.)  A  mechanism  would  also  be  available,  most  likely  on  a 
separate  screen,  to  provide  a  workspace  for  the  PV  user  to  accumu¬ 
late  views  that  are  currently  of  interest. 

Movement  through  the  program  space  is  illustrated  in  Figure  5,  which 
shows  three  displays  related  to  the  SDD-1  communication  visualiza¬ 
tion  from  Figure  1.  The  first  two  displays  were  formed  by  selecting 
transaction  modules  (TMs)  and  data  modules  (DMs) ,  as  a  PV  user  might 
do  in  localizing  a  bug.  The  third  display  is  formed  by  zooming  to 
another  level  of  detail,  to  access  a  table  of  information  about 
transactions  running  at  a  particular  TM. 

While  we  are  emphasizing  the  hierarchic  organization  of  illustra¬ 
tions  becsuse  it  is  the  dominant  organization,  there  will  actually 
be  a  network  of  links  between  the  illustrations  from  different 
cstegories.  The  complexity  of  this  subordinste  organization  poses 
two  types  of  challenges:  the  implementation  problem  of  handling  mul¬ 
tiple  connections,  and  the  knowledge  representation  problem  of  keep¬ 
ing  the  information  manageable  for  the  user.  For  the  first,  s  sur- 
fsce  must  permit  an  arbitrary  number  of  ports,  i.e.  links  to  loca- 
tions  on  other  surfsces.  For  the  second,  clean  organization  is 
important,  and  navigational  aids  must  be  provided  for  the  PV  user. 

Our  approach  to  these  problems  drsws  on  experience  with  SDMS  (111 , 
the  Spatisl  Data  Management  System.  SDMS  was  motivated  by  the  needs 
of  a  growing  community  of  people  who  need  access  to  information  in  a 
data  base  management  system  but  who  are  not  trained  in  the  use  of 
such  systems.  We  briefly  describe  some  of  the  aspects  of  SDMS  that 
are  relevant  to  PV. 
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In  SDNS,  information  is  espreasad  through  color  graphics'  with  dif¬ 
ferent  categories  of  structured  data  having  associated  icons,  i.e., 
prototypical  pictograms.  Icons  are  generated  aeni-autonatically 
fro*  the  data  base'  according  to  a  predefined  set  of  rules.  Infor¬ 
mation  retrieved  f roe  the  data  base  is  presented  in  SDNS  in  a  mul¬ 
tidimensional  framework'  so  that  the  user  can  "soon*  through  porta 
on  the  data  surface  to  see  more  detail  or  to  see  alternative  views 
of  the  data.  The  SDNS  workstation  includes  three  screens,  a  Joys¬ 
tick  for  panning  and  rooming,  touch  sensitive  screens  to  permit  the 
user  to  select  icons  by  pointing  at  them,  and  a  data  tablet  for  the 
creation  of  new  pietograms.  As  a  navigational  aid,  one  screen 
displays  a  nor Id- view  m  which  shows  an  entire  data  surface.  A 
highlighted  rectangle  on  the  world-view  map  indicates  the  portion  of 
the  data  that  ia  being  presented  in  detail  on  another  screen.  The 
rectangle  moves  as  the  use:  manipulates  the  Joy  stick  to  move  over 
the  data  surface,  one  world-view  nap  waa  shown  in  Figure  4,  Section 
2.9. 


The  program  apace  that  we  have  described  for  PV  fits  well  into  the 
multidimensional  paradigm  of  SDNS,  and  we  aspect  much  of  the  SDNS 
experience  to  be  applicable  to  PV.  The  multilayered  PV  program 
apace  la  analogous  to  the  SDNS  data  surfaces,  and  the  further  con¬ 
nections  needed  between  illustration  categories  can  be  implemented 
with  multiple  ports.  The  combined  use  of  Joy  stick,  touch  sensitive 
screens,  and  data  tablet  can  enable  the  PV  user  to  move  through  and 
augment  the  program  space  easily.  The  world-view  map  is  of  proven 
utilityi  we  do,  however,  have  more  to  say  on  this  count. 

It  is  possible  that  the  complesity  of  the  PV  environment  will 
require  other  navigational  aids  in  addition  to  the  world-view  map. 
We  are  investigating  several  different  approaches  to  helping  the  pv 
near  navigate  within  the  system,  and  to  helping  him/her  maintain  a 
sense  of  contest.  One  such  approach  is  the  use  of  a  list  of  minia- 
turisations  of  past  displays,  both  to  provide  contest  end,  when  a 
miniaturisation  is  selected,  to  provide  a  means  to  ‘pop*  the  stack 
beck  to  a  past  view. 

We  have,  then,  a  dominant  hierarchical  organisation  with  a  subordi¬ 
nate  network  organisation,  realised  ia  a  hierarchy  of  surfaces  con¬ 
taining  multiple  porta.  We  nest  take  a  closer  look  at  the  specifi¬ 
cation  of  11 lust rations,  both  in  program  apace  and  in  the  user's 
workspace. 


I.I  Debase lag  drapbloal  Faculties 

As  a  comprehensive  environment,  it  is  crucial  that  the  PV  system  be 
easy  to  ana  or ,  where  it  must  be  more  demanding,  that  the  system 
repay  the  investment,  graphics  makes  this  Imperative  all  the  more 
e ha  1 long lag.  Designers  and  progtanmers  want  access  to  information 
about  a  program  without  having  to  spend  a  great  deal  of  time  speci¬ 
fying  haw  the  information  is  to  be  displayed. 
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W«  htvi  identified  three  wyi  in  which  graphics  facilities  can  he 
enhanced i 


1.  Incorporation  of  domain- independent  higher  level  constructs 

2.  Incorporation  of  graphical  and  test  constructa  specific  to 
prog r seeing 

3.  Automated  layout 

Work  in  each  of  these  areas  has  ieportant  inpllcations  both  for  get¬ 
ting  information  into  the  PV  environaant  and  getting  it  out.  Kach 
area  is  discussed  briefly. 

First  on  our  list  was  the  incorporation  of  dcaain- independent  higher 
level  of  constructs.  KETCBPAD  1351,  PYGMALION  (31)  •  the  SDNS  FAINT 
prograa  1121,  and  others,  have  pereltted  picture  production  without 
programming,  by  combination  of  shapes  and  free  drawing.  These  tech¬ 
niques  can  be  enhanced.  The  graphical  vocabulary  need  not  be  res¬ 
tricted  to  geometric  shapes  but  also  can  Include  higher  level  organ¬ 
isational  devices,  such  as  overlays  and  split  screens.  Froductlon 
of  animation  can  move  away  from  frame  by  frame  specification  to 
specification  by  combining  high  level  dynamic  constructs.  For  exam¬ 
ple,  a  growing  and  then  shrinking  arrow  used  to  represent  communica¬ 
tion  between  modules  should  be  specifiable  as  a  unit.  (One  approach 
to  this  problem  is  given  in  (171.)  Throughout,  defaults  for  graphi¬ 
cal  conventions  can  be  more  fully  esploited. 

As  powerful  as  high  level  domain- independent  graphical  constructs 
can  be,  we  believe  that  the  PV  environment  will  also  have  to  provide 
graphical  (and  teatual)  constructa  specific  to  programming.  For 
those  tines  when  prograsmers  must  dynamically  specify  displays,  we 
espect  the  predominant  mode  of  interaction  to  ho  the  instantiation 
of  templates,  not  the  creation  of  now  pictures  from  scratch.  Pro- 
graamets  will  have  both  default  representations  for  a  wide  range  of 
data  structures  and  the  means  to  change  these  defaults  to  meet  spe¬ 
cial  needs,  (dee,  e.g.,  1221.)  Templates  for  standard  programming 
atilties  (e.g.,  stacks,  memory  maps)  will  also  be  provided.  The 
templates  meed  not  be  limited  to  single  structures  such  as  stacks! 
we  are  exploring  the  use  of  composite  templates  that  combine  infor¬ 
mation  that  is  generally  useful  for  particular  types  of  programming. 
Finally,  for  those  eases  in  which  totally  novel  pictures  are 
required,  the  user  can  turn  to  s  graphics  editor  that  will  be 
included  is  the  PV  environment. 

Once  the  PV  uaer  selects  the  components  of  a  display,  automated  lay¬ 
out  ean  arrange  the  diagrams  an  the  display  area.  For  layout,  we 
intend  to  build  upon  work  done  as  part  of  the  SUMS  project.  Addi¬ 
tional  work  will  be  needed  to  handle  the  more  richly  connected,  and 
often  heterogeneous,  pictures  that  will  be  eeonoa  for  PV. 

no  throe  facilities  listed  —  high  level  demale-  independent  graphi¬ 
cal  language,  graphical  eons tracts  specific  to  progrsnming,  and 
automated  layout  —  can  permit  the  PV  user  to  produce  vlcualisatiemo 
with  a  minimum  of  effort,  and  without  being  inundated  in  detail. 
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3.9  Instrumenting  Programs 

The  central  focus  in  the  PV  project  is  on  providing  sn  onvironaont 
that  runs  prog cans,  not  noroly  one  that  nanipulataa  static  informa¬ 
tion.  Traditional  debuggers  give  us  some  models  and  a  set  of 
esperiences  to  draw  on.  but  there  are  ways  that  they  can  be  improved 
in  terms  of  bringing  the  level  of  interaction  with  the  debugger  up 
to  the  conceptual  level  of  the  user.  He  outline  briefly  our  philoso¬ 
phy. 

First,  the  user's  goal  la  to  aee  information  about  the  running  pro¬ 
gram.  The  use  of  two  or  more  windows  on  the  screen  to  display  a 
running  program  la  one  approach  that  has  been  used  successfully 
(I3t'3d).  For  programs  designad  to  communicate  with  a  terminal, 
one  window  simulates  the  terminal  screen.  Characters  typed  by  the 
user  appear  there  and  output  from  the  program  is  directed  there. 
Meanwhile,  other  windows  can  be  used  to  esamine  the  operation  of  the 
system.  lets  of  windows  can  also  be  used  to  display  multiple 
processes  or  to  display  causes  and  effects. 

He  intend  to  esperiment  with  the  multiple  window  approach  and  other 
modes  of  display  as  well.  In  setting  up  the  display  environment,  we 
are  paying  particular  attention  to  displaying  choices  that  have  been 
made  by  the  user.  This  is  in  hopes  of  remedying  one  of  the  problems 
with  traditional  debuggers  —  the  difficulty  of  keeping  track  of  the 
constraints  that  the  user  has  set  19  as  a  filter  for  viewing  the 
behavior  of  the  program. 

Given  a  display  environment,  the  user's  problem  is  to  specify  what 
la  to  be  displayed.  The  use  of  high  level  graphics  constructs, 
including  programming  construct  templates,  was  discussed  in  the  pre¬ 
vious  subsection.  These  templates  must  also  be  instantiated,  l.e. 
tied  to  the  cede  or  other  representations  of  the  program.  For  this, 
debugging  statements  and  code  will  be  conceptually  separate  enti¬ 
tles,  so  the  user  will  not  be  thinking  in  terms  of  altering  code 
(and  later  having  to  remove  debugging  statements).  Pointing  (graph¬ 
ical  cursors  and/or  touch  sensitive  screens)  will  be  used  exten¬ 
sively  to  specify  instantiations  of  tho  templstes.  Many  slots  in 
the  templates  can  be  filled  by  pointing  at  variables  in  the  code. 
The  programmer  can  point  to  locations  in  program  structure  diagrams 
or  typeset  listings  to  specify  areas  of  interest.  When  these  areas 
are  active,  the  program  is  slowed  down  to  a  visible  speed,  it  other 
times  it  runs  at  normal  speed,  unobserved  but  much  faster.  Simi¬ 
larly,  breakpoint  locations  can  be  indicated  by  pointing. 

The  mechanics  of  coding  the  display  specification  and  producing  the 
information  desired  from  the  running  program  have  been  left  almost 
entirely  to  the  FV  system.  This  is  the  ares  about  which  we  have  the 
least  to  say  right  new,  but  about  which  we  eapect  to  have  more  to 
aay  in  the  coming  months.  In  brief,  cor  philosophy  is  that  whether 
code  is  compiled  or  interpreted  should  be  transparent  to  the  pro¬ 
grammer.  Further,  we  hope  to  stay  out  of  the  compiler  writing  busi¬ 
ness.  Vs  Intend  to  use  existing  compllsr(s),  perhaps  sugmented  to 
ssve  information  that  sight  otherwise  be  thrown  sway  (e.g. ,  a  full 
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syabol  table) .  Unix  facilities  that  allow  on#  process  to  vain  con¬ 
trol  of  another  process  between  atateaents  will  be  exploited  (19) . 
Finally,  we  are  exploring  the  tradeoff  between  accuaulating  his¬ 
tories  of  prograa  execution  as  opposed  to  facilitating  rerunning  of 
prograas,  particularly  as  this  tradeoff  is  affected  by  persistent 
data. 

•a* 

We  base  outlined  three  areas  of  particular  concern  in  the  design  of 
a  prograa  viaual itation  aysteai  Integration  of  inforaatlon  about  the 
prograa,  design  of  high  level  graphical  facilities,  and  instruaenta- 
tion  of  the  prograa.  Readers  intereated  in  nor#  background  on  the 
FV  fraaework  are  referred  to  (13). 


4.  INFUmENTATIOM 

The  current  work  la  focused  on  the  lapleaentatlon  of  a  tool  usable 
by  Ada  prograaaers  by  1994,  and  is  proceeding  in  three  phases.  The 
various  lnpleaentationa  will  explore  a  variety  of  techniques  on  a 
powerful  high- resolution  color  display  environaent,  with  an  eye 
toward  identifying  a  useful  subset  of  techniques  which  can  be  laple- 
aented  on  a  low-cost  teralnal  coating  in  1993  wbat  an  ordinary 
alphanuaeric  teralnal  costs  today.  The  three  lapleaentatlon  phases 
are  i 

1.  The  design  of  a  visual  language  for  describing  programs, 
together  with  the  processing,  translation,  and  display  routines 
necessary  to  create  a  visualisation  of  a  prograa.  This  language 
and  its  concomitant  aachinery  is  being  developed  through 
interactions  with  prograaaers  responsible  for  large-scale  work¬ 
ing  syateaa  and  with  inf oraeti on-oriented  graphic  designers, 
ftatic  and  dynaaic  aock-upa  of  prograa  illustrations  are  being 
created  and  evaluated. 

2.  The  iapleaentation  of  a  breadboard  systea  which  Incorporates  the 
results  of  the  first  phase  and  allows  integration  with  the 
results  of  other  research  efforts,  for  nae  with  a  selected 
language  on  Unix.  It  will  be  evaluated  through  use  by  people 
aaintaining  and  developing  software  on  Unix. 

3.  The  lapleaentatlon  of  a  production  prograa  visualisation  systea 
for  Ada,  incorporating  laproveaents  identified  in  the  breadboard 
version. 


One  early  result  of  the  Phase  One  exploration  of  illustrations  la  a 
five  alnute  videotape  of  dynaaic  illustration  aock-upa  for  fBO-1 
(f roa  which  Figures  1  and  S  were  taken) .  The  feedback  on  this 
effort  baa  led  as  to  conclude  that  if  the  operation  of  a  prograa  is 
not  intuitively  obvious,  the  graphics  used  aunt  bridge  the  gap.  We 
are  therefore  developing  (and  videotaping)  another  aet  of  dynaaic 
illustrations  with  graphical  synbols  that  are  nore  perceptual  than 
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symbolic  (171)  i.e. ,  that  portray  tha  semantics  more  immediately. 


S,  SUMMARY 

This  paper ,  and  tha  project  itself,  started  fro*  tha  premise  that 
graphical  illustrations  can  make  a  significant  contribution  to  tha 
process  of  program  visualisation.  In  support  of  this  premise,  va 
first  surveyed  a  variety  of  esisting  graphical  representations  and 
diacussed  possibilities  for  estended  uses  of  graphics,  especially 
animation.  He  then  outlined  a  design  philosophy  for  an  Integrated 
graphical  programing  environment  to  support  tha  development  of 
large  systesis.  Our  focus  was,  and  is,  on  the  dynaalc  aspects  of 
program,  to  pernit  the  PV  user  to  'open  the  side  of  the  machine* 
and  watch  progress  run.  Hork  is  underway  to  provide  such  an 
environaent,  with  the  goal  of  asking  the  advantages  of  graphical 
representations  available  without  placing  an  escessive  burden  on  the 
people  responsible  for  iapleaenting  and  Baintaining  the  programs. 
He  believe  that  a  comprehensive  and  integrated  approach  to  prog  ran 
visualisation  can  have  great  impact  on  the  entire  process  of 
software  engineering,  and  on  the  cost-effective  production  and 
maintenance  of  reliable  software. 
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MAJOR  TECHNICAL  RESULTS 

The  first  half  of  FY82  was  spent  designing  the  Pro¬ 
gram  Visualization  (PV)  System  and  identifying  certain  key 
classes  of  images  that  the  system  should  support.  Partic¬ 
ular  attention  was  paid  to  the  design  of  dynamic  images  to 
illustrate  programs  as  they  run. 

This  phase  of  the  project  culminated  in  a  videotape 
produced  in  December  1981*  The  tape  was  edited  into  a 
more  concise  version  in  January  1982,  and  narration  was 
added.  Graphics  for  the  tape  were  developed  using  the 
Paint  and  Animation  subsystems  of  the  Spatial  Data  Manage¬ 
ment  System.  The  animated  images  on  the  tape  were  mock- 
ups  in  the  sense  that  they  were  not  yet  driven  by  underly¬ 
ing  software.  The  tape  shows  animated  graphic  depictions 
of  control  flow  and  data  updates.  An  important  feature  of 
the  images  that  were  designed  for  PV  is  that  they  present 
a  selection  of  levels  of  detail,  giving  the  programmer  a 
choice  of  either  an  overview  or  more  detailed  views  of  the 
system  he  or  she  is  building. 

A  copy  of  the  1982  Program  Visualization  videotape  is 
enclosed  with  this  report. 

The  second  half  of  FY82  was  spent  implementing  a  base 
level  PV  system.  During  this  time  period,  we  produced  an 
initial  version  of  the  PV  System,  with  the  following  com¬ 
ponents: 

1.  User  Interface 

The  user  sees  detailed  views  of  programs,  both  text 
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and  graphics,  through  a  system  of  multiple  overlapping 
windows. 

Commands  to  manipulate  the  information  in  these  win¬ 
dows  are  displayed  as  sets  of  buttons  on  a  separate 
menu  display. 

The  user  inputs  information  via  a  combination  of  data 
tablet,  joy  stick,  touch  sensitive  screens,  and  key¬ 
board. 

2.  Graphics  Editor 

The  PV  Graphics  Editor  allows  the  user  to  construct 
illustrations  Interactively  by  "drawing"  on  a  data 
tablet.  The  base  level  editor  allows  the  user  to 
input  shapes  and  lines  (for  connectors).  A  selection 
of  line  weights  and  fonts  permits  the  construction  of 
clear,  legible  diagrams.  Underlying  data  structures 
have  been  implemented  to  to  include  information  about 
the  structure  of  the  objects  drawn.  This  knowledge 
will  allow  upgrade  of  the  existing  editor  to  a 
knowledge-based  graphics  editor;  for  example,  when  an 
object  is  moved,  its  associated  connectors  can  be 
moved  as  well. 

3.  Text  Editor 

An  existing  text  editor,  EMACS,  was  extended  and 
integerated  into  the  system. 

4.  Binder 

A  set  of  special-purpose  routines  was  implemented  to 
enable  the  user  to  specify  the  relationships  between 
programs  and  the  graphic  depictions  of  certain  stan¬ 
dard  data  types.  The  system  currently  handles  arrays 
and  single  units  such  as  integers.  The  binding  step 
is  necessary  so  that  the  PV  system  can  know  when  the 
update  of  a  variable  in  a  program  should  cause  update 
in  an  associated  visualization.  In  the  coming  months, 
we  will  be  extending  and  generalizing  the  set  of  bind¬ 
ing  routines. 

5.  Execution  Manager 

Enhancements  have  been  designed  for  the  Execution 
Manager,  which  is  responsible  for  monitoring  code  for 
changes  that  are  relevant  to  program  visualizations 
currently  displayed.  The  new  scheme  is  a  variation  on 
tagged  memory  that  tags  UNIX  pages  instead  of  tagging 
individual  memory  locations.  The  design  uses  access 
restrictions,  setting  pages  with  variables  of  interest 
to  read-only.  When  the  system  tries  to  change  a  vari¬ 
able  of  interest,  the  access  violation  will  alert  the 
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system  to  enable  a  trace.  The  variable  can  then  go 
ahead  and  be  written,  and  the  trace  process  can  iden¬ 
tify  the  location  of  the  variable,  so  the  P V  system 
can  access  the  new  value  and  update  the  relevant  visu¬ 
alizations. 


These  capabilities  were  integrated  in  the  base  level 
implementation  of  September  1982.  At  this  time,  the  sys¬ 
tem  had  developed  far  enough  to  permit  the  creation  of  an 
animated  visualization  of  the  data  structures  of  a  sort 
subroutine.  This  visualization,  which  was  adapted  from 
Images  designed  in  the  first  half  of  the  fiscal  year,  is 
now  fully  supported  by  the  PV  system. 


TECHNOLOGICAL  SIGNIFICANCE 

We  expect  on-line,  interactive  graphics  to  have  a 
profound  impact  on  software  development,  analogous  to  the 
way  that  word  processors  have  affected  the  production  of 
text.  With  respect  to  static  graphics,  the  multi¬ 
dimensional  graphic  information  structure  that  we  are 
developing  for  PV  will  allow  the  programmer  to  move  easily 
between  pieces  of  related  information  (e.g.  requirements 
and  system  structure).  With  respect  to  dynamic  graphics, 
PV's  animated  views  of  program  execution  can  help  program¬ 
mers  achieve  a  deeper  and  more  accurate  understanding  of 
the  behavior  of  their  programs. 


PRESENTATIONS  AND  PUBLICATIONS 
December  1981 

Christopher  Herot,  Mark  Friedell,  and  Diane  Smith 
took  part  in  a  DARPA  conference  organized  by  Craig  Fields 
of  the  System  Sciences  Division.  Copies  of  the  presenta¬ 
tions  were  compiled  in  "DARPA  Conference  on  Computer 
Software  Graphics,  Key  West  Florida,  Dec.  13-15  1981." 

January  1982 

Christopher  Herot  and  Gretchen  Brown  took  part  in  the 
IFIP  WG  8.1  Working  Conference  on  Automated  Tools  for 
Information  Systems  Design  and  Development,  New  Orleans, 
26-28  January,  1982.  The  paper  presented  at  this  confer¬ 
ence,  "An  Integrated  Environment  for  Program  Visualiza¬ 
tion",  appeared  in  Schneider  and  Wasserman  (eds.), 
Automated  Tools  for  Information  Systems  Design,  North- 
Holland  Publishing  Co.,  1982. 
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February  1982 

Christopher  Herot  gave  a  presentation  on  PV  at  a 
meeting  of  the  Northeastern  ACM  Chapter  on  February  18. 

June  1982 

Mark  Friedell  presented  a  talk  on  PV  and  the  VIEW 
Project  at  the  workshop  on  Automated  Explanation  Produc¬ 
tion.  This  conference  was  sponsored  by  the  University  of 
Southern  California  Information  Sciences  Institute  and  was 
held  at  the  university's  Idylwild  Campus. 

July  1982 

Craig  Fields  and  Clint  Kelly  of  DARPA  visited  CCA  on 
July  6.  Christopher  Herot  and  Gretchen  Brown  gave  a  short 
status  report  on  the  PV  project,  and  Richard  Carling  gave 
a  demonstration  of  the  window  management  software. 
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STATUS  REPORT  (N00014-81 -C-0456 ) 
8  November  1982  -  7  February  1983 


This  report  summarizes  the  work  completed  during  the 
seventh  quarter  (8  Nov  1982  -  7  Feb  1983)  of  the  Program 
Visualization  project  under  contract  N000 14-81-0-0456. 


1.  IMPLEMENTATION  WORK 

The  first  phase  of  implementation  of  the  Program  Visu¬ 
alization  (PV)  Library  is  now  complete,  with  basic 
store  and  access  functions  supported.  In  addition, 
the  <  system  supports  automatic  library  access  of 
appropriate  code  when  the  user  points  to  a  graphic 
symbol  from  the  library.  Also  supported  for  standard 
datastructures  is  automatic  library  access  of  graphic 
depictions  when  the  user  points  to  code  he  or  she  has 
typed  in. 

We  made  incremental  progress  this  quarter  on  implemen¬ 
tation  of  the  PV  Graphic  Editor  and  further 
integration/upgrading  of  the  PV  Text  Editor. 

A  new  system  of  graphical  menus  has  been  installed. 
Considerable  design  work  went  into  organizing  the 
large  number  of  commands  provided  by  the  system.  The 
new  menus  were  produced  using  the  PV  Graphic  Editor. 

Animated  line  highlighting  is  now  supported  for  C 
code,  i.e.,  lines  of  code  are  highlighted  as  they  are 
executed. 

2.  DESIGN  OF  GRAPHIC  SYMBOLS 

We  have  been  designing  a  number  of  graphic  symbols  for 
different  parts  of  the  system.  We  now  have  a  first 
version  of  a  set  of  four  symbols  to  be  used  as  logos 
for  the  different  PV  navigational  aids,  as  well  as 
about  a  dozen  symbols  to  mark  major  categories  of 
operations  on  the  PV  menus.  Work  on  graphic  symbols 
to  signify  different  standard  datastructures  is  also 
underway. 

An  initial  version  of  the  notation  used  for  system 
architecture  diagrams  in  CCA's  Multibase  Project  is 
now  supported.  A  "kit"  of  symbols  allows  the  user  to 
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conatruct  nodes  in  this  notation  by  copying  rather 
than  drawing.  Connectors  are  drawn  by  the  user  via 
the  general  sake-connector  command. 

3.  SITE  VISITS 

Bob  Grafton  of  ONR  visited  on  December  9.  We  gave  him 
an  extended  PV  slide  presentation  and  a  demonstration 
of  the  system. 

Clint  Kelly  of  DARPA  visited  on  January  13 •  and  he 
talked  with  us  for  some  time  about  PV.  We  showed  him 
some  slides  of  the  current  state  of  the  system,  along 
with  a  demonstration. 

4.  PAPERS  AND  PRESENTATIONS 

We  were  notified  of  acceptance  of  the  position  paper 
we  submitted  to  the  ACM  SIGSOFT/SIGPLAN  Software 
Engineering  Symposium  on  High-Level  Debugging,  which 
will  be  held  in  March  of  1983.  We  also  wrote  and  sub¬ 
mitted  a  paper  to  the  ACM  IEEE  Design  Automation 
conference  to  be  held  in  June  of  1983. 

On  January  11,  We  gave  a  talk  on  PV  at  Intermetrics  as 
part  of  their  lecture  series.  There  was  a  good-sized 
audience  of  50-100  people,  and  there  seemed  to  be  a 
lot  of  interest  in  our  work. 

5.  VIDEOTAPE  COMPLETED 

A  videotape  that  demonstrates  some  key  aspects  of  the 
current  PV  implementation  was  completed  in  February. 
The  tape,  which  runs  approximately  ten  minutes,  gives 
an  overview  of  the  system  and  then  shows  how  a  pro¬ 
grammer  could  use  PV  both  to  compose  code  and  to  con¬ 
struct  an  associated  dynamic  visualization.  The 
dynamic  visualization  includes  both  highlights  moving 
through  lines  of  code  and  graphic  depictions  of  vari¬ 
ables  being  updated. 


